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77?e MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
• Any reply received by the Office later than three months after the mailing date of this communication, even if timely Tiled, may reduce any 

earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)H Responsive to communication(s) filed on 10/25/2005 . 
2a)S This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 1-15 is/are pending in the application. 

4a) Of the above claim(s) 11-15 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1-10 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 02 October 2003 is/are: a)E3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. §119 

12) S Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)[3 All b)D Some * c)D None of: 

1 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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2) □ Notice of Drafts person's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) □ Notice of Informal Patent Application (PTO-152) 
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DETAILED ACTION 



Response to Amendment 



1. 



This office action is in response to an amendment filed 10/25/2005. 



2. 



Claims 1, 3, 4, 5, 6, 7, 8, 9, and 10 have been amended. 



Claim Rejections - 35 USC § 103 



3. Claims 1 , 3, 5, and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Loscos et al. (NPL Document, "Interactive Virtual Relighting of Real Scenes", 
herein referred to as "Loscos"). 

As per claims 1 and 5, Loscos teaches the claimed "comparing the position of the 
light source and a virtual position of a display object in a three-dimensional image to 
obtain a relative positional relation therebetween" by teaching of "We then have the 
ability to select and change the intensity of real and virtual light source and also add 
virtual objects into the scene ... All modifications are interactive and all diffuse common 
illumination effects such as shadows between real and virtual objects" (paragraph below 
figure 25 on pg. 303). Illumination effects such as shadows between real and virtual 
objects would require data of the positional relationship between a virtual position of a 
display object (virtual object) and a light source in order to achieve the quoted 
functionality in the reference. 

Loscos does not explicitly teach the claimed "detecting a position of a light 
source". However, Loscos teaches of Loscos, pg. 293, 3rd paragraph under sec. 3 
"The positions of the light sources are also measured manually and inserted into the 3D 
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model" (3rd paragraph under section 3, pg. 293) where it would have been obvious to 
one of ordinary skill in the art to use a detector or detecting to measure a light source 
position. 

As per claims 3 and 6, the reasons and rationale for the rejection of claim 1 are 
incorporated herein. Loscos does not explicitly teach the claimed "detecting positions of 
a plurality of light sources" and does not explicitly teach the claimed "a plurality of 
detectors". However, Loscos teaches of Loscos, pg. 293, 3rd paragraph under sec. 3 
"positions of the light sources are also measured manually" (3rd paragraph under 
section 3, pg. 293) where it would have been obvious to one of ordinary skill in the art to 
use detectors or detecting to measure the light source positions. 

4. Claims 2 and 4 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Loscos in view of Foley ("Computer Graphics: Principles and Practice", 1996, herein 
referred to as "Foley"). 

As per claim 2, Locos teaches the claimed "the position of the virtual light source 
and the virtual position of the display object ... to obtain the relative positional relations 
therebetween" for the same reasons as the claimed comparing the position light source 
and display object step in claim 1 . Locos teaches of virtual light sources as well by 
stating "We then have the ability to select and change the intensity of real and virtual 
light source and also add virtual objects into the scene ... All modifications are 
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interactive and all diffuse common illumination effects such as shadows between real 
and virtual objects" (paragraph below figure 25 on pg. 303). 

Loscos does not teach the claimed "detecting lightness of the light source". 
Foley teaches this limitation on page 734, equation 16.20 where the I sub lambda 
values represent the intensity (lightness) of each light source. Using the Foley equation 
(16.20) would require that the system also detect the lightness value of each light 
source. 

It would have been obvious to one of ordinary skill in the art to combine Loscos 
with Foley. Loscos suggests the need for detecting lightness by stating "the designer 
can then interactively manipulate real light intensities" where such a detector would be 
convenient for getting initial lightness values before interactive manipulation by the 
designer. 

As per claim 4, Loscos does not teach the claimed "obtaining a position of a 
single virtual light source, which represents the plurality of light sources". Foley teaches 
these limitations on page 734, equation 16.20 and states "If there are m light sources, 
then the terms for each light source are summed" (first sentence under section 16.1.6, 
pg. 734). The process of summing each light source produces the end result of a virtual 
light source at a position representing the light coming each individual light source. 

It would have been obvious to one of ordinary skill in the art to combine Loscos 
with Foley. Loscos would benefit from such a combination by simplifying the plurality of 
light sources into one representative light source, which could result in faster processing 
and calculations. 
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1. Claims 7-10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Loscos in view of Benton (US Patent 6917370, herein referred to as "Benton"). 

As per claims 7-9, Loscos teaches the claimed "display surface configured to 
display the three-dimensional image" by teaching of a "graphics hardware to display" 
(pg. 301 , second paragraph above section 7.2) and by teaching of "3D representation of 
a real scene" (4 th paragraph under section 1). 

Loscos does not teach the claimed "detector is disposed on at least one of the 
display surface and a surface adjacent to the display surface" and "detector is disposed 
to be adjacent to the display surface". Benton teaches the claimed detector in figure 5, 
piece 520, which shows a portable camera (detector) and a display piece 590. Benton 
does not explicitly teach of "disposed the detector adjacent to the display". However, it 
would have been obvious to one of ordinary skill in the art to dispose the detector 
adjacent to the display in order to achieve a natural view and perspective for the 
operator of what lies in their visual site path (i.e. using the detector to display the 
overlaid graphics in figure 4 for navigation). 

Loscos does not teach the claimed "detector is disposed at a. position where the 
detector which detects the light source from the light in the same direction as at least 
one of a display direction of the three dimensional image and a direction in which the 
three-dimensional image is observed". The user of the system of Benton can be sailing 
in a direction (see figure 4) where the sun (light source) is in the same direction as 
onboard camera (figure 5, piece 520) for overlaying graphics where the detector 
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(camera) detects light in the same direction as the 3D image is observed (graphic image 
in front on boat as seen in figure 4, also see col 5, lines 48-50). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine Loscos and 
Benton. The light coming in from the same direction from which the image is observed 
(and also as viewed from the display) can very likely have the biggest impact and 
influence of the scene as viewed by the user. Thus, placing a sensor in this direction to 
detect the incoming light would be a convenient and simple way to make all such light is 
recorded and used in the mixed reality system. 

As per claim 10, Loscos does not teach the claimed "detector includes three- 
primary-colors detection means for adding colors to the shade". Benton teaches this 
limitation in figure 5, piece 520, which shows a camera (detector) and where it is known 
that such a camera is capable of detecting primary colors. 

Response to Arguments 

2. The objections to the title, specification, and claims have been withdrawn in 
response to this amendment. 

3. The objection to the foreign priority has been withdrawn in response to applicant 
providing certified copies of the missing documents. 

4. Applicant's arguments filed 10/25/2005 have been fully considered but they are 
not persuasive. 
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Applicant in section IV is arguing limitations amended after the writing of the 
previous office action. This office action has been changed to address the new 
limitations. 

Applicant in section V is arguing limitations amended after the writing of the 
previous office action and further argues references individually in the 35 USC 103(a) 
rejections in regards to Foley and Benton. 

5. In response to applicant's arguments against the references individually, one 
cannot show nonobviousness by attacking references individually where the rejections 
are based on combinations of references. See In re Keller, 642 F.2d 413, 208 
USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 
1986). 

Conclusion 

6. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Daniel F. Hajnik whose telephone number is (571 ) 272- 
7642. The examiner can normally be reached on Mon-Fri (8:30A-5:00P). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka J. Chauhan can be reached on (571) 272-7782. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

DFH 



■ ULKA CHAUHAN 

SUPERVISORY PATENT EXAMINER 



